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ABSTRACT 


Computers are becoming ubiquitous and are playing significant roles 
in our lives. Domestic digital devices for leisure and entertainment 
are becoming increasingly important. To be usable, every computing 
device must allow for some form of interaction with its user. The 
human-computer interaction is the point of communication between 
the human user and the computer. AI has been gradually being 
incorporated into human-computer interaction (HCI). As AI systems 
become more and more ubiquitous, it is imperative to understand 
those systems from a human perspective. This paper provides an 


introduction to the “marriage” between HCI and AI. 
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INTRODUCTION 

The earliest user interface to a computer was the 
punched card devised by Joseph Marie Jacquard in 
1804 [1]. Computers have been interacting with 
humans as restaurant servers, customer service 
representatives, health care providers, and sexual 
partners. In 2011, 15-million people watched on TV 
as an IBM computer, named Watson, beat human 
contestants [2]. 


Although people have interacted with computers from 
the beginning, it took a while for human—computer 
interaction (HCI) to be recognized as a field. HCI is 
concerned with understanding how people interact 
with machines in general and with computing systems 
in particular. It is the study of how people design 
computer systems and how computers affect 
individuals and society. It focuses on the interfaces 
between human users and computing devices [3]. HCI 
was formerly known as the man-machine studies or 
man-machine interaction. It is an ever expanding 
multidisciplinary field which sits at the intersection 
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between social sciences (on the human side) and 
computer science (on the computer side). 


Artificial intelligence (AI) is the use of computer 
science to develop machine that can be trained 
to learn, reason, communicate, and make human-like 
decisions. It is a technology that is rapidly being 
adopted in many industries to improve performance, 
precision, time efficiency, and cost reduction. 
Although AI is a branch of computer science, there is 
hardly any field which is unaffected by this 
technology. Common areas of applications include 
agriculture, business, law enforcement, oil and gas, 
banking and finance, education, transportation, 
healthcare, automobiles, entertainment, 
manufacturing, speech and text recognition, facial 
analysis, and telecommunications [4]. Today, the 
fields of AI and HCI are converging. 


As shown in Figure 1, the relationship between AI 
and HCI falls somewhere into the spectrum between 
human-computer interaction and intelligent systems 
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[5]. Although some view AI and HCI as having 
opposing views of how humans and computers should 
interact, we believe in unifying AI and HCI. The 
underlying divide is not between AI and HCI, but 
between rationalistic and design approach [6]. The 
two fields explore the nexus of computing and 
intelligent behavior. Although several factors work to 
keep the fields of AI and HCI apart, the AI and HCI 
communities need to make AI ready for human 
interaction. AI and HCI can mutually benefit from a 
closer collaboration. 


This chapter provides a brief introduction to 
application of artificial intelligence on human- 
computer interaction. It begins by providing a general 
overview on both artificial intelligence and human- 
computer interaction. Then, it presents the various 
applications of combining the two fields. It discusses 
the benefits and challenges of AI and HCI. The last 
section concludes with comments. 


OVERVIEW ON ARTIFICIAL INTELLIGENCE 
The term “artificial intelligence” (AI) was coined in 
1956 by John McCarthy during a conference held on 
this subject. Al is the branch of computer science that 
deals with designing intelligent computer systems that 
mimic human intelligence. The ability of machines to 
process natural language, to learn, to plan makes it 
possible for new tasks to be performed by intelligent 
systems. The main purpose of AI is to mimic the 
cognitive function of human beings and perform 
activities that would typically be performed by a 
human being. AI is_ stand-alone independent 
electronic entity that functions much like human 
expert. Today, AI is integrated into our daily lives in 
several forms, such as personal assistants, automated 
mass transportation, aviation, computer gaming, 
facial recognition at passport control, voice 
recognition on virtual assistants, driverless cars, 
companion robots, etc. [7]. 


An important feature of AI technology is that is can 
be added to existing technologies. AI has benefited 
many areas such chemistry and medicine, where 
routine diagnoses can titiated by Al-aided computers. 
It embraces a wide range of disciplines such as 
computer science, engineering, machine learning, 
chemistry, biology, physics, astronomy, neuroscience, 
and social sciences. 


AI is not a single technology but a range of 
computational models and algorithms. The major 
disciplines in AI include expert systems, fuzzy logic, 
and artificial neural networks (ANNs), machine 
learning, deep learning, natural language processing, 
computer vision, and robotics. The various computer- 
based tools or technologies that have been used to 
achieve AI’s goals are the following [8,9]: 


> Expert Systems: An expert system (ES) (or 
knowledge-based system) enables computers to 
make decisions by interpreting data and selecting 
between alternatives just as a human expert would 
do. It uses a technique known as rule-based 
inference in which rules are used to process data. 


> Neural Networks: These computer programs 
identify objects or recognize patterns after having 
been trained. Artificial neural networks (ANNs) 
are parallel distributed systems consisting of 
processing units (neurons) that calculate some 
mathematical functions. The ANN _ model 
represents nonlinear relationships which are 
directly learned from the data being modeled. 
Neural networks are being explored for healthcare 
applications in imaging and diagnoses, risk 
analysis, lifestyle management and monitoring, 
health information management, and virtual 
health assistance. 


> Natural Language Processors: Computer 
programs that translate or interpret language as it 
is spoken by normal people. Natural language 
processors (NLP) techniques extract information 
from unstructured data. NLP targets at extracting 
useful information from the narrative text to assist 
decision making. NLP includes applications such 
as speech recognition, text analysis, translation 
and other goals related to language. There are two 
basic approaches to NLP: statistical and semantic. 
Healthcare is the biggest user of the NLP tools. 
NLP has been used in the clinical setting for 
capturing, representing, and utilizing clinical 
information [10]. 


> Robots: Computer-based programmable 
machines that have physical manipulators and 
sensors. For example, medical robots can help 
with surgical operations, rehabilitation, social 
interaction, assisted living, etc. Robotic-guidance 
is becoming common in spine surgery [11]. 


> Fuzzy Logic: Reasoning based on imprecise or 
incomplete information in terms of a range of 
values rather than point estimates. Fuzzy logic 
deals with uncertainty in knowledge that 
simulates human reasoning in incomplete or fuzzy 
data. The fuzzy model is robust to parameter 
changes and tolerant to impression. 


> Machine Learning: Algorithms to make 
predictions and interpret data and “learn”, without 
static program instructions. ML is a statistical 
technique for fitting models to data and training 
models with data. Machine learning (ML) extracts 
features from input data by constructing analytical 
data algorithms and examines the features to 
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create predictive models. The most common ML 
algorithms are supervised learning, unsupervised 
learning, reinforcement learning, and deep 
learning. Machine learning is widely used in 
human-computer interaction [12]. 


> Deep Learning: A subset of machine learning 
built on a deep hierarchy of layers, with each 
layer solving different pieces of a complex 
problem. It aims at increasing the capacity of 
supervised and unsupervised learning algorithms 
for solving complex real-world problems by 
adding multiple processing layers. An illustration 
of deep learning with two hidden layers is in 
Figure 2 [13]. Figure 3 shows how AI 
encompasses machine learning and deep learning 
[14]. 


> Data Mining: This deals with the discovery of 
hidden patterns and new knowledge from large 
databases. Data mining exhibits a variety of 
algorithmic tools such as statistics, regression 
models, neural networks, fuzzy sets, and 
evolutionary models. 


Each AI tool has its own advantages. Using a 
combination of these models, rather than a single 
model, is recommended. AI technologies are 
drastically influencing the retail industry and 
customer experience. An emerging area of interest in 
AI is to make AI agents cooperate with each other. 


FUNDAMENTALS OF HUMAN-COMPUTER 
INTERACTION 

As shown in Figure 4 [15], human—computer 
interaction (HCI) draws on psychology, social and 
cultural anthropology, human — geography, 
ergonomics, artificial intelligence, and potentially any 
discipline that deals with human behaviors. While 
computer science is the hard science of HCI, 
psychology is its soft science [16]. The term “human- 
computer interaction” was popularized by Stuart K. 
Card, Allen Newell, and Thomas P. Moran in their 
1983 book, The Psychology of Human-Computer 
Interaction. The main goal of HCI is to produce 
usable and safe computer systems. Usability (or 
usefulness) is often defined in terms of efficiency, 
effectiveness and satisfaction. The success of any 
product is determined by the satisfaction of the 
humans who use the product. The data that affect 
usability of a system include age, experience, and 
capabilities. 


As its name suggests, human-computer interaction 

(HCI) consists of three parts: 

> The Human: This refers to an individual user or 
a group of users. The underlying objective of HCI 
is that people using a computer system should 


come first. The design of HCI should be user- 
centered or user-friendly. This requires careful 
analysis of user characteristics and the goals that 
users want to achieve. Humans have emotion and 
emotion can influence important their behaviors 
including technology use and customer loyalty. 
While natural language input by the human 
appears desirable in human-computer 
communication, this assumes that the human can 
express his knowledge and that the computer can 
interpret the language. 


> The Computer: This refers to any computing 
technology ranging from desktop computers, to 
large scale computer systems. Before we used to 
deal with stand-alone computers, but today 
computers are embedded in several devices. 
Devices such as mobile phones or VCRs may also 
be regarded as computers. The rapid development 
of computers has made effective HCI essential. 


> The Interaction: Humans interact with 
computers in several ways using different 
interfaces such as keyboards, mice, tablets, voice, 
and hand gestures. For example, desktop 
applications use handheld computers graphical 
user interfaces (GUI), while speech recognition 
systems use voice user interfaces (VUI). In a 
world where the user interface has been mostly 
visual, focus has been on the GUL. The interaction 
may occur at the physical, social, and 
psychological context. Many other paradigms for 
human-computer interaction exist. Perhaps one of 
the best-known paradigms is the World Wide 
Web with the browser and HTML 


We must understand these parts in order to 
understand HCI. Applications of HCI include 
information kiosks, computer terminals, interactive 
Web sites, computer games, mobile devices, robots, 
computer games, NLP, eye-tracker, hand gesture, 
fatigue monitoring, improving electronic medical 
records (EMR), natural language interfaces, human 
speech recognition, and surgery [17]. 


APPLICATIONS OF AI IN HCI 

The AI-related technologies are silently redefining the 
way we make our intentions known to computers. 
The fields of AI and HCI are mutually influencing as 
never before. The synergies of Aland HCL, especially 
interactive machine learning, could become very 
relevant in several areas such as medical and business 
applications. Some common applications of AI in 
human-computer interaction include computer vision, 
speech recognition, natural language processing [18]: 


> Natural Language Processing (NLP): This s is 
the technology that enables computers to 
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understand human speech as it is spoken. NLP is 
drastically changing the way we interact with 
more complex systems. Natural communication 
among humans consists primarily of a 
combination of speech, hand gestures, facial 
expressions, eye motions, and body language. 
Gestures are not equivalent to speech, but they 
complement each other. Conversational AI 
systems like Alexa are constantly growing 
smarter and better at language and conversation, 
all made possible through advances in machine 
learning. By combining data from speech and 
non-verbal signals computers may  learm 
appropriate patterns of interactions for 
cooperation between humans and computers [19]. 


> Multimodal Interaction: This is part of 
everyday human discourse. We speak, move, and 
gesture in an effective flow of communication. To 
improve reliability and usability, researchers are 
now designing multimodal interfaces that 
automatically learn and adapt to important and 
environmental parameters [20]. Artificial 
Intelligence (AI) is powering speech-based and 
gesture-based systems, helping to fundamentally 
change HCI from simple, unimodal channels to 
complex, multimodal channels. 


> Computer Vision: With the progress in deep 
learning and neural networks, we have made great 
inroads in the field of computer vision. AI and 
computer vision have made eye tracking more 
efficient and affordable. 


> Computer Emotion: The recent developments in 
human-computer interaction have emphasized 
user-centered rather than computer-centered 
approach. This led to design of intelligent and 
affective interfaces. Important discoveries in 
neuroscience and psychology have contributed to 
an interest in the scientific study of emotion. 
Without the ability to process such affect or 
emotion information, computers cannot be 
expected to communicate with humans in a 
natural way. The ability of machines to recognize, 
interpret and respond according to human 
emotions is known as adaptive or affective 
computing. It is computing that relates to emotion 
or other affective phenomena. It involves the 
study and development of systems and devices 
that can recognize and express human affects or 
emotions. It uses both hardware and software 
technology to detect the affective state of a person 
[21]. 


> Computer Games: Computer games are 
becoming popular and the encounter between 
games and culture has become inevitable. 


Applying HCI skills would enable game 
developers to design applications that are tailored 
to the player’s cultural background, allowing 
developers to transcend cultural barriers [22]. 
Combining the human-computer interaction and 
the artificial intelligence areas can promote 
accessibility in games. This will lead them 
making them more democratic and useful for 
society. 


BENEFITS 

Human-computer interfaces have benefits both in 
terms of performance and in terms of the human's role 
in the system. The computer will do whatever it is 
commanded to do. It has no ego or emotion and never 
feels overworked or underworked [23]. HCI makes a 
big difference in terms of efficiency, convenience, 
even for life and death. The world is full of badly 
designed products and some products look good but 
may not work well. The skills developed with HCI 
help us to understand why some products are good 
and others are bad. There are some disabled people 
who cannot control parts of their bodies and real time 
human —computer interaction systems can help them 
[24]. 


The AI revolution is coming at a rapid speed, 
promising better performance than human 
professionals. AI does not need to sleep, rest, food, or 
need air to breathe. It can predict human interactions 
with computers and identify patterns far quicker than 
humans. It is not just replacing jobs (the takeover of 
human jobs by automation), it is creating new jobs as 
well. We must decide whether to ride on the wave of 
Alor resist the inevitable change. The combination of 
Aland HCI can improve education and make homes 
smarter. 


CHALLENGES 

AI and HCI have both potential benefits and risks. 
The biggest challenge product designers have is 
escaping the mindset of their own _ technical 
understanding and gaining a perspective of what it is 
like to be a user. HCI is a rapidly evolving discipline. 
Other challenges involved in HCI design include 
performing real evaluations, tailoring software, and 
recognizing design conflicts [25]. Several factors 
impede precision in HCI devices due to a lack of 
standard for the quality of data obtained by such 
devices. Courses offered on HCI must be carefully 
put together so that they do not quickly become out of 
date. 


The potential negative effects of AI on society cannot 
be ignored. Merely introducing human guidance into 
AI applications is not enough to prevent its 
unintended consequences. Human involvement in AI 
system design and development is critical to ensure 
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that Al-based systems work as intended. Some AI 
applications were very expensive and failed due to a 
lack of use value. It would be a security threat if AI 
could not be fully controlled by humans. Although AI 
is advancing rapidly, ethics and security are not [26]. 


CONCLUSION 

Much of the research effort in HCI seeks to improve 
human-computer interaction by improving the 
usability of human-computer interfaces. It is a young 
field with considerable success. Researchers have 
begun to be more concerned about the need for a 
stronger “marriage” between HCI and AI. 


As AI systems become more and more ubiquitous, it 
is imperative to understand those systems from a 
human perspective. This will help the designers figure 
out how to build collaborative relationships between 
humans and computers to enhance human creativity. 
The future HCI will definitely shape around AI. A 
new field of “Human Artificial Intelligence 
Interaction” will emerge from this. For this reason, 
HCI has gradually become part of the curricula in 
several institutions [27]. The HCI courses should 
incorporate AI technology. Allowing AI to 
revolutionize education will mean that it will take less 
time for students to learn new skills after graduation. 
What the future holds is anyone’s guess. 


More information about HCI can be found in several 
books in [28-40] and several others available on 
Amazon.com. One should also consult the following 
journals exclusively devoted to AI and HCI: 

> Human Computer Interaction 

> Computers in Human Behavior 

>» ACM _ Transactions on Computer-Human 
Interaction 

Artificial Intelligence 

Applied Artificial Intelligence 
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Figure 1 The relationship between AI and human-computer interaction [5]. 





Figure 2 An illustration of deep learning with two hidden layers [13]. 





Figure 3 Artificial intelligence encompasses machine learning and deep learning [14]. 
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Figure 4 Human-computer interaction and related research fields [15]. 
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